A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E).

BACKGROUND: Observational epidemiologic studies provide critical data for the evaluation of the potential effects of environmental, occupational and behavioural exposures on human health. Systematic reviews of these studies play a key role in informing policy and practice. Systematic reviews should incorporate assessments of the risk of bias in results of the included studies. OBJECTIVE: To develop a new tool, Risk Of Bias In Non-randomized Studies - of Exposures (ROBINS-E) to assess risk of bias in estimates from cohort studies of the causal effect of an exposure on an outcome. METHODS AND RESULTS: ROBINS-E was developed by a large group of researchers from diverse research and public health disciplines through a series of working groups, in-person meetings and pilot testing phases. The tool aims to assess the risk of bias in a specific result (exposure effect estimate) from an individual observational study that examines the effect of an exposure on an outcome. A series of preliminary considerations informs the core ROBINS-E assessment, including details of the result being assessed and the causal effect being estimated. The assessment addresses bias within seven domains, through a series of 'signalling questions'. Domain-level judgements about risk of bias are derived from the answers to these questions, then combined to produce an overall risk of bias judgement for the result, together with judgements about the direction of bias. CONCLUSION: ROBINS-E provides a standardized framework for examining potential biases in results from cohort studies. Future work will produce variants of the tool for other epidemiologic study designs (e.g. case-control studies). We believe that ROBINS-E represents an important development in the integration of exposure assessment, evidence synthesis and causal inference.

Expanded Systematic Evidence Map for Hundreds of Per- and Polyfluoroalkyl Substances (PFAS) and Comprehensive PFAS Human Health Dashboard.

BACKGROUND: Per- and polyfluoroalkyl substances (PFAS) encompass a class of chemically and structurally diverse compounds that are extensively used in industry and detected in the environment. The US Environmental Protection Agency (US EPA) 2021 PFAS Strategic Roadmap describes national research plans to address the challenge of PFAS. OBJECTIVES: Systematic Evidence Map (SEM) methods were used to survey and summarize available epidemiological and mammalian bioassay evidence that could inform human health hazard identification for a set of 345 PFAS that were identified by the US EPA's Center for Computational Toxicology and Exposure (CCTE) for in vitro toxicity and toxicokinetic assay testing and through interagency discussions on PFAS of interest. This work builds from the 2022 evidence map that collated evidence on a separate set of ∼150 PFAS. Like our previous work, this SEM does not include PFAS that are the subject of ongoing or completed assessments at the US EPA. METHODS: SEM methods were used to search, screen, and inventory mammalian bioassay and epidemiological literature from peer-reviewed and gray literature sources using manual review and machine-learning software. For each included study, study design details and health end points examined were summarized in interactive web-based literature inventories. Some included studies also underwent study evaluation and detailed extraction of health end point data. All underlying data is publicly available online as interactive visuals with downloadable metadata. RESULTS: More than 13,000 studies were identified from scientific databases. Screening processes identified 121 mammalian bioassay and 111 epidemiological studies that met screening criteria. Epidemiological evidence (available for 12 PFAS) mostly assessed the reproductive, endocrine, developmental, metabolic, cardiovascular, and immune systems. Mammalian bioassay evidence (available for 30 PFAS) commonly assessed effects in the reproductive, whole-body, nervous, and hepatic systems. Overall, 41 PFAS had evidence across mammalian bioassay and epidemiology data streams (roughly 11% of searched chemicals). DISCUSSION: No epidemiological and/or mammalian bioassay evidence were identified for most of the PFAS included in our search. Results from this SEM, our 2022 SEM on ∼150 PFAS, and other PFAS assessment products from the US EPA are compiled into a comprehensive PFAS dashboard that provides researchers and regulators an overview of the current PFAS human health landscape including data gaps and can serve as a scoping tool to facilitate prioritization of PFAS-related research and/or risk assessment activities. https://doi.org/10.1289/EHP13423.

I-REFF diagrams: enhancing transparency in systematic review through interactive reference flow diagrams.

Systematic review methods are recognized for their rigor and transparency and are widely adapted to frameworks that cover review types such as systematic reviews, scoping reviews, and systematic evidence maps. Reporting guidelines help promote better systematic review practices and detailed documentation of the review process for different types of health research (e.g., PRISMA-Preferred Reporting Items for Systematic Reviews and Meta-Analyses; CONSORT-Consolidated Standards of Reporting Trials; and STROBE-Strengthening the Reporting of Observational Studies in Epidemiology). Transparency in the systematic review process and reporting of results is one of the key advantages of the methods and particularly important for hazard and risk assessments due to the high level of scrutiny these reviews face from scientific, political, and public communities. Data visualizations are important to clearly convey information from a review by helping readers perceive, understand, and assess the displayed information easily and quickly. The study flow diagram is a required element of a systematic review and maps out the number of included and excluded records identified, and the reasons for exclusion. Static literature flow diagrams help viewers readily understand the general review methodology and summarize the number of records included or excluded at each stage of the review. However, such diagrams can be time-consuming to develop and maintain during a systematic review or scoping review, and they provide limited summary-level information. We explored how the use of online systematic review tools such as DistillerSR coupled with visualization software such as Tableau can efficiently generate an Interactive REFerence Flow (I-REFF) diagram that is linked to the literature screening data, thus requiring minimal preparation, and resulting in a simplified process for updating the diagram. Furthermore, I-REFF diagrams enhance transparency and traceability by not only summarizing the records in the review but also allowing viewers to follow specific records throughout the review process. We present an example I-REFF diagram and discuss recommendations for key interactive elements to include in these diagrams and how this workflow can improve efficiency and result in an accessible and transparent interactive literature flow diagram without advanced programming.

An Approach to Assessing the Influence of Environmental and Occupational Cancer Hazard Identification on Policy Decision-Making.

BACKGROUND: Cancer hazard identification is critical to informing decisions on preventive actions. However, the influence of cancer hazard assessments on the creation of health-protective regulations is poorly understood. Although prior studies have measured the health and economic benefits of regulatory actions in general, we are not aware of efforts to explicitly study the influence of cancer hazard identification on policy decisions in the United States. OBJECTIVES: In this commentary, we present an approach to examine whether formal identification of a substance as a human carcinogen may prompt a regulatory action to reduce exposure to carcinogens and enhance public health. Further, we discuss the broader implications of cancer hazard identification on policy decision-making, including identifying gaps and providing recommendations. METHODS: Using the Report on Carcinogens (RoC) as a test case, we systematically searched U.S. federal and state databases for notices of regulations mentioning the RoC from 1995 to 2023. For each regulation, we extracted information on the carcinogen(s) regulated, the regulatory agency, the regulatory purpose, the economic sector exposure sources, and the analyzed public health benefits and costs. We created a publicly available, web-based interactive tool to visualize the data. DISCUSSION: U.S. regulatory agencies have been using cancer hazard evaluations, such as the RoC, for decades to inform public health policy actions to prevent or mitigate cancer risks. Specifically, nonregulatory cancer hazard assessments have been used to prioritize chemical evaluations, support regulatory-based assessments, and trigger regulatory action. Our approach showed that assessing the influence of cancer hazard identification on science-based public health policies is feasible, informative, and needed, and our study is a first step in this direction. We recommend expanding this approach to other cancer and noncancer hazard assessments to ultimately inform our understanding of the influence of hazard classifications on policymaking. https://doi.org/10.1289/EHP12681.

Systematic evidence mapping informs a class-based approach to assessing personal care products and pubertal timing.

BACKGROUND: Personal care products (PCPs) contain many different compounds and are a source of exposure to endocrine disrupting chemicals (EDCs), including phthalates and phenols. Early-life exposure to EDCs commonly found in PCPs has been linked to earlier onset of puberty. OBJECTIVE: To characterize the human and animal evidence on the association between puberty-related outcomes and exposure to PCPs and their chemical constituents and, if there is sufficient evidence, identify groups of chemicals and outcomes to support a systematic review for a class-based hazard or risk assessment. METHODS: We followed the OHAT systematic review framework to characterize the human and animal evidence on the association between puberty-related health outcomes and exposure to PCPs and their chemical constituents. RESULTS: Ninety-eight human and 299 animal studies that evaluated a total of 96 different chemicals were identified and mapped by key concepts including chemical class, data stream, and puberty-related health outcome. Among these studies, phthalates and phenols were the most well-studied chemical classes. Most of the phthalate and phenol studies examined secondary sex characteristics and changes in estradiol and testosterone levels. Studies evaluating PCP use and other chemical classes (e.g., parabens) had less data. CONCLUSIONS: This systematic evidence map identified and mapped the published research evaluating the association between exposure to PCPs and their chemical constituents and puberty-related health outcomes. The resulting interactive visualization allows researchers to make evidence-based decisions on the available research by enabling them to search, sort, and filter the literature base of puberty-related studies by key concepts. This map can be used by researchers and regulators to prioritize and target future research and funding to reduce uncertainties and address data gaps. It also provides information to inform a class-based hazard or risk assessment on the association between phthalate and phenol exposures and puberty-related health outcomes.

Evaluating endocrine disrupting chemicals: A perspective on the novel assessments in CLARITY-BPA.

BACKGROUND: The Consortium Linking Academic and Regulatory Insights on Bisphenol A Toxicity (CLARITY-BPA) was a collaborative research effort to better link academic research with governmental guideline studies. This review explores the secondary goal of CLARITY-BPA: to identify endpoints or technologies from CLARITY-BPA and prior/concurrent literature from these laboratories that may enhance the capacity of rodent toxicity studies to detect endocrine disrupting chemicals (EDCs). METHODS: A systematic literature search was conducted with search terms for BPA and the CLARITY-BPA participants. Relevant studies employed a laboratory rodent model and reported results on 1 of the 10 organs/organ systems evaluated in CLARITY-BPA (brain and behavior, cardiac, immune, mammary gland, ovary, penile function, prostate gland and urethra, testis and epididymis, thyroid hormone and metabolism, and uterus). Study design and findings were summarized, and a risk-of-bias assessment was conducted. RESULTS: Several endpoints and methods were identified as potentially helpful to detect effects of EDCs. For example, molecular and quantitative morphological approaches were sensitive in detecting alterations in early postnatal development of the brain, ovary, and mammary glands. Hormone challenge studies mimicking human aging reported increased susceptibility of the prostate to disease following developmental BPA exposure. Statistical analyses for nonmonotonic dose responses, and computational approaches assessing multiple treatment-related outcomes concurrently in linked hormone-sensitive organ systems, reported effects at low BPA doses. CONCLUSIONS: This review provided an opportunity to evaluate the unique insights provided by nontraditional assessments in CLARITY-BPA to identify technologies and endpoints to enhance detection of EDCs in future studies.

Epidemiology Evidence for Health Effects of 150 per- and Polyfluoroalkyl Substances: A Systematic Evidence Map.

BACKGROUND: Per- and polyfluoroalkyl substances (PFAS) comprise a large class of chemicals with widespread use and persistence in the environment and in humans; however, most of the epidemiology research has focused on a small subset. OBJECTIVES: The aim of this systematic evidence map (SEM) is to summarize the epidemiology evidence on approximately 150 lesser studied PFAS prioritized by the EPA for tiered toxicity testing, facilitating interpretation of those results as well as identification of priorities for risk assessment and data gaps for future research. METHODS: The Populations, Exposure, Comparators, and Outcomes (PECO) criteria were intentionally broad to identify studies of any health effects in humans with information on associations with exposure to the identified PFAS. Systematic review methods were used to search for literature that was screened using machine-learning software and manual review. Studies meeting the PECO criteria underwent quantitative data extraction and evaluation for risk of bias and sensitivity using the Integrated Risk Information System approach. RESULTS: 193 epidemiology studies were identified, which included information on 15 of the PFAS of interest. The most commonly studied health effect categories were metabolic (n=37), endocrine (n=30), cardiovascular (30), female reproductive (n=27), developmental (n=26), immune (n=22), nervous (n=21), male reproductive (n=14), cancer (n=12), and urinary (n=11) effects. In study evaluation, 120 (62%) studies were considered High/Medium confidence for at least one outcome. DISCUSSION: Most of the PFAS in this SEM have little to no epidemiology data available to inform evaluation of potential health effects. Although exposure to the 15 PFAS that had data was fairly low in most studies, these less-studied PFAS may be used as replacements for "legacy" PFAS, leading to potentially greater exposure. It is impractical to generate epidemiology evidence to fill the existing gaps for all potentially relevant PFAS. This SEM highlights some of the important research gaps that currently exist. https://doi.org/10.1289/EHP11185.

Systematic evidence map (SEM) template: Report format and methods used for the US EPA Integrated Risk Information System (IRIS) program, Provisional Peer Reviewed Toxicity Value (PPRTV) program, and other "fit for purpose" literature-based human health analyses.

BACKGROUND: Systematic evidence maps (SEMs) are gaining visibility in environmental health for their utility to serve as problem formulation tools and assist in decision-making, especially for priority setting. SEMs are now routinely prepared as part of the assessment development process for the US Environmental Protection Agency (EPA) Integrated Risk Information System (IRIS) and Provisional Peer Reviewed Toxicity Value (PPRTV) assessments. SEMs can also be prepared to explore the available literature for an individual chemical or groups of chemicals of emerging interest. OBJECTIVES: This document describes the typical methods used to produce SEMs for the IRIS and PPRTV Programs, as well as "fit for purpose" applications using a variety of examples drawn from existing analyses. It is intended to serve as an example base template that can be adapted as needed for the specific SEM. The presented methods include workflows intended to facilitate rapid production. The Populations, Exposures, Comparators and Outcomes (PECO) criteria are typically kept broad to identify mammalian animal bioassay and epidemiological studies that could be informative for human hazard identification. In addition, a variety of supplemental content is tracked, e.g., studies presenting information on in vitro model systems, non-mammalian model systems, exposure-level-only studies in humans, pharmacokinetic models, and absorption, distribution, metabolism, and excretion (ADME). The availability of New Approach Methods (NAMs) evidence is also tracked (e.g., high throughput, transcriptomic, in silico, etc.). Genotoxicity studies may be considered as PECO relevant or supplemental material, depending on the topic and context of the review. Standard systematic review practices (e.g., two independent reviewers per record) and specialized software applications are used to search and screen the literature and may include the use of machine learning software. Mammalian bioassay and epidemiological studies that meet the PECO criteria after full-text review are briefly summarized using structured web-based extraction forms with respect to study design and health system(s) assessed. Extracted data is available in interactive visual formats and can be downloaded in open access formats. Methods for conducting study evaluation are also presented which is conducted on a case-by-case basis, depending on the usage of the SEM.

Assessing author willingness to enter study information into structured data templates as part of the manuscript submission process: A pilot study.

Background: Environmental health and other researchers can benefit from automated or semi-automated summaries of data within published studies as summarizing study methods and results is time and resource intensive. Automated summaries can be designed to identify and extract details of interest pertaining to the study design, population, testing agent/intervention, or outcome (etc.). Much of the data reported across existing publications lack unified structure, standardization and machine-readable formats or may be presented in complex tables which serve as barriers that impede the development of automated data extraction methodologies.As full automation of data extraction seems unlikely soon, encouraging investigators to submit structured summaries of methods and results in standardized formats with meta-data tagging of content may be of value during the publication process. This would produce machine-readable content to facilitate automated data extraction, establish sharable data repositories, help make research data FAIR, and could improve reporting quality. Objectives: A pilot study was conducted to assess the feasibility of asking participants to summarize study methods and results using a structured, web-based data extraction model as a potential workflow that could be implemented during the manuscript submission process. Methods: Eight participants entered study details and data into the Health Assessment Workplace Collaborative (HAWC). Participants were surveyed after the extraction exercise to ascertain 1) whether this extraction exercise will impact their conducting and reporting of future research, 2) the ease of data extraction, including which fields were easiest and relatively more problematic to extract and 3) the amount of time taken to perform data extractions and other related tasks. Investigators then presented participants the potential benefits of providing structured data in the format they were extracting. After this, participants were surveyed about 1) their willingness to provide structured data during the publication process and 2) whether they felt the potential application of structured data entry approaches and their implementation during the journal submission process should continue to be further explored. Conclusions: Routine provision of structured data that summarizes key information from research studies could reduce the amount of effort required for reusing that data in the future, such as in systematic reviews or agency scientific assessments. Our pilot study suggests that directly asking authors to provide that data, via structured templates, may be a viable approach to achieving this: participants were willing to do so, and the overall process was not prohibitively arduous. We also found some support for the hypothesis that use of study templates may have halo benefits in improving the conduct and completeness of reporting of future research. While limitations in the generalizability of our findings mean that the conditions of success of templates cannot be assumed, further research into how such templates might be designed and implemented does seem to have enough chance of success that it ought to be undertaken.

Systematic Evidence Map for Over One Hundred and Fifty Per- and Polyfluoroalkyl Substances (PFAS).

BACKGROUND: Per- and polyfluoroalkyl substances (PFAS) are a large class of synthetic (man-made) chemicals widely used in consumer products and industrial processes. Thousands of distinct PFAS exist in commerce. The 2019 U.S. Environmental Protection Agency (U.S. EPA) Per- and Polyfluoroalkyl Substances (PFAS) Action Plan outlines a multiprogram national research plan to address the challenge of PFAS. One component of this strategy involves the use of systematic evidence map (SEM) approaches to characterize the evidence base for hundreds of PFAS. OBJECTIVE: SEM methods were used to summarize available epidemiological and animal bioassay evidence for a set of ∼150 PFAS that were prioritized in 2019 by the U.S. EPA's Center for Computational Toxicology and Exposure (CCTE) for in vitro toxicity and toxicokinetic assay testing. METHODS: Systematic review methods were used to identify and screen literature using manual review and machine-learning software. The Populations, Exposures, Comparators, and Outcomes (PECO) criteria were kept broad to identify mammalian animal bioassay and epidemiological studies that could inform human hazard identification. A variety of supplemental content was also tracked, including information on in vitro model systems; exposure measurement-only studies in humans; and absorption, distribution, metabolism, and excretion (ADME). Animal bioassay and epidemiology studies meeting PECO criteria were summarized with respect to study design, and health system(s) were assessed. Because animal bioassay studies with ≥21-d exposure duration (or reproductive/developmental study design) were most useful to CCTE analyses, these studies underwent study evaluation and detailed data extraction. All data extraction is publicly available online as interactive visuals with downloadable metadata. RESULTS: More than 40,000 studies were identified from scientific databases. Screening processes identified 44 animal and 148 epidemiology studies from the peer-reviewed literature and 95 animal and 50 epidemiology studies from gray literature that met PECO criteria. Epidemiological evidence (available for 15 PFAS) mostly assessed the reproductive, endocrine, developmental, metabolic, cardiovascular, and immune systems. Animal evidence (available for 40 PFAS) commonly assessed effects in the reproductive, developmental, urinary, immunological, and hepatic systems. Overall, 45 PFAS had evidence across animal and epidemiology data streams. DISCUSSION: Many of the ∼150 PFAS were data poor. Epidemiological and animal evidence were lacking for most of the PFAS included in our search. By disseminating this information, we hope to facilitate additional assessment work by providing the initial scoping literature survey and identifying key research needs. Future research on data-poor PFAS will help support a more complete understanding of the potential health effects from PFAS exposures. https://doi.org/10.1289/EHP10343.